Learning and Using Hand Abstraction Values for Parameterized Poker Squares

We describe the experimental development of an AI player that adapts to different point systems for Parameterized Poker Squares. After introducing the game and research competition challenge, we describe our static board evaluation utilizing learned evaluations of abstract partial Poker hands. Next, we evaluate various time management strategies and search algorithms. Finally, we show experimentally which of our design decisions most significantly accounted for observed performance. Introduction The inaugural EAAI NSG Challenge1 was to create AI to play a parameterized form of the game Poker Squares. Our best approach first applies Monte Carlo -greedy reinforcement learning to the task of learning a static (i.e. heuristic) evaluation function that estimates the expected final score for an abstraction of the game state. It then plays the game with expectimax search using this static evaluation at a shallow depth cutoff. In this section, we describe the game of Poker Squares and the parameterization of the game. In the next section, we focus on different approaches to the static evaluation of nonterminal game states and experimental results comparing simple greedy play with such approaches. We then turn our attention to time management, i.e. the apportionment of search time during real-time play. Finally, we compare different search techniques, gaining insight to which work best for this problem domain.